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ABSTRACT 


The  Air  Force  JOVIAL  Compiler  Validation  System  (JCVS)  was 
developed  to  assist  in  the  validation  of  performance  of  proposed 
JOVIAL  language  processors.  This  report  discusses  the  JCVS  in  the 
context  of  its  use  by  the  Air  Force  Directorate  of  ADPE  Selection. 
It  provides  a  certification  of  the  audit  test  modules,  and  makes 
recommendations  for  improvements  to  the  JOVIAL  audit  capability. 

It  is  recommended  that  the  audit  programs  be  used  independently  of 
the  JCV  System. 
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SECTION  I 


INTRODUCTION 


The  Air  Force  procedure  for  the  competitive  procurement  of 
general  purpose  automatic  data  processing  equipment  requires  the 
validation  of  the  performance  claims  made  for  each  proposed  ADP  system 
in  order  to  establish  the  responsiveness  to  the  systems  requirements 
of  the  Request  for  Proposal.  Among  these  requirements,  there  fre¬ 
quently  occurs  the  need  for  a  JOVIAL  language  processor  which  must 
conform  to  the  AFM  100-24  JOVIAL  (J3)  Language  Specifications. 

The  Air  force  JOVIAL  Compiler  Validation  System  (JCVS)  (2)  waS  developed 
under  contract  to  assist  in  the  task  of  validation  of  the  performance 
of  proposed  JOVIAL  language  processors.  This  report  briefly  discusses 
the  JCVS  in  the  context  of  its  possiole  use  by  the  Air  Force  Direc¬ 
torate  of  ADPE  Selection,  MCS ,  summarizes  the  experience  acquired  in 
working  with  the  JCVS,  critiques  the  audit  test  modules,  and  makes 
recommendations  for  improvements  to  the  JOVIAL  audit  capability. 

The  compiler  validation  process  or  audit  function  has  two 
purposes:  the  first  is  to  establish  that  each  language  feature  of 

the  Air  Force  standard  programming  language  is  accepted  by  the  JOVIAL 
processor  under  examination;  the  second  is  to  establish  that  the 
execution  of  each  language  feature  produces  the  prescribed  results. 

The  JCVS  accomplishes  this  audit  function  by  presenting  to  the  JOVIAL 
processor  a  set  of  one  or  more  JOVIAL  source  programs  and  their 
required  input  data  (if  any) ,  which  together  contain  statements 
invoking  each  of  the  required  standard  JOVIAL  features.  Such  programs 
are  referred  to  as  "Audit  Programs."  Upon  successful  compilation 
and  execution  of  the  audit  programs,  it  is  established  that  a  processor 
does  conform  to  the  standard  for  those  features  tested.  While  it  is 
impossible  for  the  audit  programs  to  check  for  the  correct  implemen¬ 
tation  of  all  possible  combinations  of  standard  features  due  to  the 
large  number  of  tests  that  would  be  required,  it  is  expected  that  a 
sufficiently  representative  sample  will  be  included  to  establish 
reasonable  confidence  in  the  capability  of  the  observed  system. 
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SECTION  II 


THE  JCVS  COMPONENTS 


The  heart  of  the  Air  Force  JCVS  is  the  Population  File.  It 
contains  all  of  the  JOVIAL  source  statements  comprising  the  tests  of 
the  standard  features  and  the  auxiliary  procedures  for  reporting  the 
results  of  execution  of  the  tests.  In  addition  to  the  Population 
File,  the  JCVS  consists  of  a  Population  File  Maintenance  program,  a 
Selector  program,  a  Source  Program  Maintenance  program,  and  a  con¬ 
version  program  for  character  set  transformation.  These  auxiliary 
programs  are  written  in  ANS  COBOL  and  provide  the  means  for  automating 
the  manipulation  of  the  constituent  elements  of  the  JOVIAL  audit 
programs . 


THE  POPULATION  FILE 

The  Population  File  (Pop  File)  is  the  data  base  for  the  JCVS. 

It  contains  approximately  9000  cards  of  JOVIAL  source  statements 
and  commentary  which  are  subdivided  into  test  modules  of,  at  most,  50 
cards  each.  Each  test  module  contains  all  of  the  JOVIAL  statements 
necessary  to  effect  the  test  of  a  given  feature  including  all  item 
declarations,  procedure  declarations,  etc.  The  test  module  is 
completely  self-contained,  with  the  exception  of  the  output  procedures, 
which  are  invoked  to  record  the  results  of  the  test.  The  report 
writing  or  output  procedures  are  contained  in  a  separate  module, 
providing  a  uniform  reporting  mechanism.  Thus ,  each  test  module 
as  it  exists  in  the  Pop  File  may  be  compiled  and  executed  independ¬ 
ently  of  every  other  test  module. 

The  test  modules  may  also  be  compiled  and  executed  together  as 
a  single  JOVIAL  program  by  combining  the  individual  modules  in  any 
desired  sequence.  Through  the  use  of  appropriate  conventions  in 
assigning  names  to  variables,  labels,  procedures,  etc.,  duplication 
of  names  has  been  avoided.  The  characteristics  of  the  J3  language 
are  such  that  no  rearrangement  or  sorting  of  the  JOVIAL  statements 
of  the  test  modules  is  necessary.* 


*In  this  regard  the  J3  language  differs  significantly  from  both 
FORTRAN  and  COBOL.  In  FORTRAN  specification  statements  must  precede 
all  executable  statements,  and  in  COBOL  appropriate  statements  must 
appear  in  the  ENVIRONMENT,  DATA  and  PROCEDURE  divisions  of  the  program 
in  that  order  (cf  MTR-1953?The  Air  Force  COBOL  Compiler  Validation 
System  (CCVS))# 
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The  physical  arrangement  of  the  Pop  File  consists  of  4000 
characters,  fixed  length  records,  each  containing  the  characters  from 
fifty  80-character  cards.  The  first  card  of  each  record  is  a  header 
card  which  identifies  the  module  and  may  also  designate  a  dependence 
upon  other  specific  modules.  The  header  card  is  in  the  form  of  a 
JOVIAL  comment  line  and  is  considered  to  be  a  part  of  the  JOVIAL 
test  module.  If  more  than  50  cards  are  required  for  a  test,  there 
is  provision  on  the  header  card  to  specify  that  the  module  is  to 
be  appended  to  the  preceding  module  as  a  continuation  of  it.  Finally, 
the  header  card  may  designate  that  the  module  is  to  be  present  in 
every  audit  program,  whether  or  not  its  selection  has  been  requested. 
Characters  73-76  of  every  card  contain  the  module  identification  num¬ 
ber,  and  characters  78-80  contain  the  card  sequence  number  within 
the  module,  which  must  be  in  the  range  of  1  to  50  inclusive.  The 
77th  character  of  the  header  card  designates  which  of  the  foregoing 
functions  is  intended  for  that  module.  On  all  other  cards,  this 
character  designates  JOVIAL  source  statements.  The  module  identifi¬ 
cation  number,  the  card  sequence  number  and  the  function  character 
provide  the  key  basis  for  the  functional  manipulations  performed 
upon  the  Pop  File  by  the  other  components  of  the  JCV  System. 

The  first  record  of  the  Pop  File  has  the  module  identification 
number  001,  and  fulfills  additional  unique  functions.  The  first  five 
cards  of  that  record  may  be  used  to  convey  to  the  user  for  informa¬ 
tional  purposes  only  data  concerning  the  source  of  the  particular 
Pop  File  and  the  hardware  environment  in  which  it  may  operate.  The 
content  of  these  cards  is  ignored  by  the  JCVS.  Succeeding  cards  in 
the  first  record  are  designated  environmental  software  cards,  and 
may  be  operating  system  control  cards,  JOVIAL  processor  control  cards, 
etc.,  which  are  to  be  placed  before  and/or  after  every  audit  program 
when  it  is  selected  from  the  Pop  File.  By  offsetting  these  environ¬ 
mental  software  cards  by  one  character,  the  JCVS  is  able  to  handle 
operating  system  control  cards  which  require  special  characters  in 
Column  1, 


the  population  file  maintenance  program 

The  Population  File  Maintenance  Program  (P0PFM)  provides  a 
mechanism  for  creating  a  new  Pop  File  either  from  cards  for  the 
individual  modules,  or  by  updating  an  old  Pop  File  by  means  of  dele¬ 
tion,  replacement  or  insertion  of  either  individual  cards  or  entire 
modules  on  the  basis  of  the  module  identification  numbers  and  card 
sequence  numbers.  A  report  of  the  functions  performed  is  generated 
by  P0PFM. - 
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THE  SELECTOR  PROGRAM 


The  JCVS  Selector  Program  (SELECT)  operates  on  the  Pop  File  to 
produce  a  single  JOVIAL  source  program  from  one  or  more  JOVIAL  test 
modules  residing  on  the  Pop  File.  The  user  designates,  by  means  of 
control  cards,  those  modules  which  are  to  be  selected.  The  selected 

modules  appear  in  order  by  module  number  as  they  exist  on  the  Pop 
File.  The  user  may  direct  the  selected  modules  to  be  written  onto 
magnetic  tape  or  punched  cards.  An  audit  file  is  produced  by  SELECT 
containing  a  record  of  the  modules  selected  and  additional  messages 
relating  to  suspected  error  conditions  encountered. 


THE  SOURCE  PROGRAM  MAINTENANCE  PROGRAM 

The  JCVS  Source  Program  Maintenance  Program  (SOPMM)  operates  on 
an  existing  JCVS  Source  Program  File  to  update  and  generate  a  new 
Source  Program  File.  The  SOPMM  provides  the  user  with  the  ability  to 
add  information  to  the  source  program  file,  delete  information  from 
the  source  program  file,  or  replace  information  on  the  source  program 
file  on  a  card  image  by  card  image  basis,  according  to  the  module 
identification  number  and  the  card  sequence  number.  The  user  may 
direct  the  new  source  program  file  either  to  magnetic  tape  or  punched 
cards.  An  audit  file  is  produced  optionally  by  SOPMM  providing  a 
record  of  the  diagnostic  and  trace  messages  as  well  as  a  source 
program  listing  when  desired. 


THE  POPULATION  FILE  INITIATING  PROGRAM 

The  JCVS  Pop  File  Initiating  Program  (INIPOP)  operates  to 
initiate  and,  at  the  user’s  option,  to  re-number  a  Pop  File  either 
from  an  existing  Pop  File  or  from  a  card  file  containing  the  test 
modules.  The  module  identification  numbers  only  are  re-sequenced  as 
directed  by  user  supplied  control  cards.  Certain  cross-references 
between  modules  are  changed  automatically  when  affected  by  the  re¬ 
numbering  process.  An  audit  report  is  created  which  contains 
diagnostic  messages,  and  an  optional  listing  of  JOVIAL  source  state¬ 
ments  on  the  new  Pop  File.  A  punched  card  deck  of  the  Pop  File  may 
be  obtained  as  an  optional  output. 
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THE  REPORT  WRITER  PROGRAM 


The  JCVS  Report  Writer  P rogram  (JCVSRP)  operates  on  the  Pop 
File  to  produce  a  listing  of  the  test  modules  on  the  Pop  File  or 
a  listing  of  all  the  test  header  cards  on  the  Pop  File,  or  both  as 
directed  by  user  supplied  control  cards. 

(2) 

The  reader  is  referred  to  the  JCVS  User’s  Manual  for  a  more 
detailed  description  of  the  JCVS  components  and  their  use.  In  the 
work  with  the  JCVS  Test  Modules  which  was  performed  on  both  the 
GE-635  at  RADC  and  the  IBM/360-50  at  MITRE,  it  was  found  more  con¬ 
venient  to  manipulate  the  test  module  card  decks  manually,  rather 
than  to  use  the  JCVS  programs  enumerated  above.  The  JCVS  programs 
require  that  control  cards  be  prepared  in  addition  to  the  JOVIAL 
statement  modifications,  their  use  requires  additional  computer  runs 
to  be  made  to  generate  the  modified  audit  programs  before  they  can 
be  processed  by  the  JOVIAL  compiler,  and  the  reruns  occasioned  by 
errors  introduced  in  these  peripheral  activities  lengthen  the 
apparent  turn  around  time  to  process  an  audit  program  with  the  JOVIAL 
compiler.  However,  the  JCVS  components  have  each  been  used  at  least 
once,  and  they  appear  to  function  as  described.  It  was  found 
necessary  to  introduce  a  few  minor  modifications  to  initialize  print 
records  and  repair  clerical  errors. 
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SECTION  III 


THE  JCVS  POPULATION  FILE 


The  present  set  of  JCVS  Test  Modules  contained  in  the  Population 
File  fall  into  four  general  categories: 

1.  The  Pass/Fail  Reporter  and  its  verification  tests 

2.  Tests  of  Declarations 

3.  Tests  of  Procedural  Statements 

4.  Tests  of  Processing  Declarations 

Each  of  these  categories  of  tests  is  discussed  in  this  section 
with  respect  to  its  operational  characteristics,  the  thorouhgness 
and  rigor  of  test  logic,  and  the  completeness  of  coverage  of  the 
AFM  100-24  JOVIAL  features. 


PASS /FAIL  REPORTER 

The  Pass/Fail  Reporter  consists  of  three  JOVIAL  Procedures 
contained  in  Test  Module  9998.  Their  function  is  to  print  one  line 
identifying  the  test  module  and  a  second  line  indicating  that  the 
test  was  successful  or  had  failed.  The  Procedure  OUTERR  prints  a 
line  by  calling  a  FORTRAN  subroutine  in  which  the  information  to  be 
printed  is  the  40-character  Hollerith  argument.  This  FORTRAN  sub¬ 
routine  is  not  provided,  nor  are  any  specifications  for  it  given.  It 
is  not  obvious  how  a  standard  conforming  FORTRAN  program  could  be 
made  to  manipulate  successfully  this  Hollerith  argument. 

The  procedure  OUTERA  is  invoked  at  the  beginning  of  execution 
of  each  test  to  set  up  the  test  module  identification  line.  The 
4-character  Hollerith  argument  of  this  procedure  is  the  identifi¬ 
cation  number  of  the  test,  which  is  then  combined  with  the  invariant 
part  of  the  message  by  use  of  the  JOVIAL  OVERLAY  feature.  The 
message  is  printed  by  invoking  the  procedure  OUTERR. 

At  the  conclusion  of  its  execution,  each  test  module  invokes 
the  procedure  OUTERB  to  set  up  and  print  the  Pass/Fail  message.  The 
argument  of  the  procedure  OUTERB  is  a  1-character  Hollerith  variable 
which  is  assumed  to  have  been  set  to  the  value  "Y"  if  the  test  were 
successful,  and  to  "N"  if  the  test  failed.  The  EQ  relational  opera¬ 
tor  is  used  in  an  IF: c lause  to  compare  the  argument  with  the  Hollerith 
literal  constant  1H(Y) .  If  the  condition  is  true,  the  40-character 
Hollerith  variable  is  assigned  the  value  of  the  "test  successful" 
message,  and  OUTERR  is  invoked  to  print  it.  If  the  condition  is  not 
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true,  then  the  variable  is  assigned  the  value  of  the  ’’test  failed” 
message  and  the  procedure  OUTERR  is  invoked  by  the  same  statement. 

The  Pass/Fail  Reporter  provides  a  uniform  reporting  mechanism 
for  all  tests,  and  localizes  all  output  statements  and  interfaces 
with  FORTRAN  in  this  one  module.  This  particular  embodiment  offers 
some  disadvantages.  The  use  of  other  than  standard  JOVIAL  output 
procedures  makes  the  Audit  Programs  processor  dependent,  so  that  they 
must  be  specifically  adapted  to  each  processor  to  be  tested.  The 
FORTRAN  interface  with  a  Hollerith  argument  introduces  a  dependency 
upon  the  processor  under  test.  The  Pass/Fail  Reporter  is  complicated 
by  the  use  of  nested  procedures,  i.e.  procedures  invoking  other  pro¬ 
cedures;  by  the  use  of  argument  passing  between  procedures;  by  the 
use  of  logic  within  the  Reporter;  and  by  the  use  of  the  OVERLAY 
feature.  Thus,  the  Pass/Fail  Reporter  is  more  sophisticated  and  com¬ 
plex  in  terms  of  JOVIAL  features  employed  than  many  of  the  tests 
which  use  it  to  report  their  success  or  failure,  and  might  itself  be 
a  source  of  error,  preventing  the  use  of  any  of  the  audit  programs. 

The  use  of  the  OVERLAY  feature  might  introduce  processor  dependencies 
due  to  word  boundaries  in  data  which  are  to  be  written  out.  Finally, 
the  Pass/Fail  Reporter  provides  no  information  on  the  test  results 
which  can  be  used  either  to  confirm  the  success  or  to  assist  in 
diagnosing  the  failure  of  a  test. 

The  function  of  the  five  test  modules  0500-0520  is  to  validate 
the  Pass/Fail  Reporter.  As  originally  designed,  these  modules 
attempted  to  verify  the  correct  functioning  of  a  processor  for  each 
of  the  JOVIAL  language  features  which  are  employed  in  the  Reporter. 

To  do  this  independently  of  the  Reporter  itself,  non-standard  JOVIAL 
output  statements  were  employed  to  print  the  test  results.  These 
statements  had  been  subsequently  replaced  with  calls  to  the  Pass/ 

Fail  Reporter  procedures  for  printing,  with  the  result  that  the 
desired  independent  verification  of  these  features  has  not  been 
achieved.  A  further  defect  in  these  tests  is  that  the  features 
presumed  to  be  tested  are  not  tested  under  the  same  conditions  of 
use  in  the  Pass/Fail  Reporter.  For  example,  the  Pass/Fail  Reporter 
uses  an  IF: clause  having  a  Hoi leri th : 1 i teral : rela tion  list  as  its 
boolean: formula,  but  in  test  nodule  0520  the  IFrclause  feature  is 
tested  using  only  the  boolean: constants  0  and  1  for  the  boolean: for¬ 
mula.  Certainly  different  machine  code  is  required  for  the  relational 
comparison  of  literal  formulas  than  for  boolean: cons tants ,  and  the 
success  of  the  test  of  the  latter  case  does  not  imply  that  the  pro¬ 
cessor  will  treat  the  former  correctly.  Further,  a  sophisticated 
optimizing  processor  might  recognize  the  invariance  of  the  boolean: 
cons  tants  and  generate  code  which  would  ignore  completely  the  condi¬ 
tional  statement  as  employed  in  test  module  0520,  thus  completely 
invalidating  the  purpose  of  the  test. 
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The  following  JOVIAL  features  are  those  presumed  to  be  tested 
by  modules  0500-0520: 

1.  Hollerith  Item  Description 

2.  Preset  Hollerith  Item  , 

3.  Procedure  Call  using  Hollerith  Argument 

4.  Hollerith  Assignment  Statement 

5.  GOTO  Statement 

6.  IF : c lause ,  boolean: cons tant 

7.  Procedure  Definition,  with  no  Argument 

8.  Procedure  Call,  with  no  Argument 

The  Pass/Fail  Reporter  Module  9998  uses  all  of  these  features 
except  the  last  two  items,  and  in  addition,  employs  the  OVERLAY 
feature,  the  definition  of  a  procedure  having  a  Hollerith  argument, 
and  the  nesting  of  procedures  with  argument  passing  required.  These 
last  features  are  not  explicitly  validated  by  test  modules  0500-0520. 

A  few  specific  programming  errors  were  noted  in  module  9998. 

The  global  variables  0UT1,  OUTA,  OUTB  were  not  defined.  Since  these 
variables  are  referenced  in  almost  every  module,  it  is  necessary 
that  their  item: descriptions  occur  at  the  beginning  of  the  audit 
programs.  The  argument  AA  of  the  procedure  OUTERR(AA)  was  not  passed 

to  the  output  subroutine,  so  that  the  message  to  be  printed  was  lost.  s 

Finally,  in  the  definition  of  the  procedure  OUTERA(CC) ,  its  argument 
was  illegally  declared  to  OVERLAY  another  datum.  Since  at  execution 

time  the  dummy  argument  CC  is  replaced  by  any  number  of  actual  t 

arguments,  each  of  which  may  be  defined  in  a  different  calling  pro¬ 
gram  unit,  such  use  of  OVERLAY  is  inconsistent. 

In  view  of  the  difficulties  enumerated  above,  MITRE  provided 
a  new  module  0198  to  replace  test  module  9998  as  the  Pass/Fail 
Reporter.  This  module  is  listed  in  Appendix  I.  The  procedure  names 
and  arguments  have  been  retained  to  be  compatible  with  the  existing 
procedure  calls,  but  each  procedure  is  now  independent  of  the  others; 
i.e.  there  is  no  nesting  of  procedures  and  concommitant  argument 
passing.  Global  variables  are  used  to  transmit  information  to  the 
procedures  rather  than  relying  on  the  procedure  arguments.  The  use 
of  the  OVERLAY  feature  was  retained,  but  it  now  occurs  at  the  global 
level,  rather  than  within  the  output  procedures,  and  it  does  not 
involve  the  procedure  arguments.  This  could  be  avoided  by  using  a 
larger  variety  of  print  files  to  accommodate  the  different  message 
lengths,  by  making  all  data  for  output  the  same  length,  (e.g.  OUTA, 

OUTB  and  0UT1) ,  or  by  introducing  a  more  complex  feature  such  as  an 
array  or  table  for  constructing  the  output  message.  Short  of 
revising  the  entire  mechanism,  it  was  felt  that  this  represents  the 
best  compromise.  A  further  modification  was  made  to  cause  the 
value  of  the  Pass/Fail  indicator  to  be  printed  as  part  of  the  Pass/ 

Fail  Message,  and  to  reset  this  parameter  to  the  failed  condition, 
i.e.  0UTA=1H(N) ,  after  each  success  or  fail  message  is  printed.  As 
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a  result  the  next  test  module  must  set  the  parameter  to  "Y",  other¬ 
wise  the  default  condition  would  be  that  the  test  failed. 

Also  provided  is  a  new  test  module  0100  to  validate  this  Pass/ 
Fail  Reporter.  Rather  than  attempt  to  validate  independently  each 
of  the  features  employed  therein,  it  assumes  that  all  of  the  features 
are  implemented  correctly,  and  attempts  to  demonstrate  that  each  of 
the  three  procedures  does  indeed  produce  the  expected  results  when 
called  with  appropriate  parameter  values.  One  line  is  printed 
directly  by  a  JOVIAL  OUTPUT  statement  in  this  test  module  when  the 
print  file  is  opened.  A  second  line  is  printed  by  a  call  to  the 
procedure  OUTERR,  and  a  third  line  results  from  a  call  to  the  pro¬ 
cedure  OUTERA,  which  incorporates  the  four-character  module  number 
into  the  test  header  line.  Then  to  test  the  Pass/Fail  function,  the 
procedure  OUTERB  is  called,  first  with  the  parameter  OUTA  set  to 
"N",  then  again  after  setting  OUTA  to  "Y"  to  demonstrate  that  the 
appropriate  pass  and  fail  messages  are  printed  in  each  instance. 

Then,  OUTERB  is  called  a  third  time  to  demonstrate  that  the  para¬ 
meter  OUTA  had  been  reset  to  "N"  after  printing  the  "test  successful'’ 
message.  The  test  module  then  prints  a  final  line  using  a  direct 
JOVIAL  OUTPUT  statement,  indicating  the  end  of  the  test  and  the 
expected  number  of  lines  which  are  printed.  The  auditor  must  examine 
the  output  and  determine  that  the  appropriate  messages  have  been 
printed  in  the  expected  sequence. 

The  JOVIAL  file: declaration  which  defines  the  print  file  is 
included  in  the  test  module  0100,  as  are  also  the  item: descriptions 
for  the  global  parameters  used  by  the  audit  programs.  It  is  there¬ 
fore  necessary  that  modules  0100  and  0198  must  be  present  in  every 
audit  program,  and  that  they  should  be  positioned  at  the  beginning 
of  the  program.  Hence,  the  relatively  low  module  numbers  which  have 
been  given  them.  The  AFM  100-24  provides  that  the  device: name 
occurring  in  the  file: declaration  is  to  be  defined  by  each  implemen¬ 
tor.  It  is  therefore  to  be  expected  that  this  statement  which  occurs 
only  once  in  module  0100  will  have  to  be  changed  for  each  processor 
to  be  tested. 

In  testing  the  JCVS  with  the  J3  processor  on  the  TBM  system/ 360 
Model  50,  the  efficacy  of  the  revised  Pass/Fail  Reporter  and  valida¬ 
tion  procedure  was  demonstrated,  when  it  was  found  that  the  "test 
successful"  message  was  always  printed,  regardless  of  whether  the 
parameter  OUTA  had  the  value  "Y"  or  "N".  This  was  due  to  a  pro¬ 
cessor  deficiency,  which  would  not  have  been  discovered  by  the 
original  verification  procedure. 


9 


DECLARATIONS 


There  are  151  test  modules  which  are  designed  to  test  the 
declaration  features  of  the  JOVIAL  language.  Each  test  references 
the  appropriate  section  of  AFM  100-24  which  describes  the  feature 
being  tested,  and  the  set  of  data  declarations  for  simple  items, 
arrays  and  tables  of  all  types  appears  to  be  reasonably  complete 
with  respect  to  the  variety  of  permissible  forms  which  are  represented. 

The  item  declarations  are  the  means  for  associating  names  with 
elements  of  a  program  and  for  describing  the  attributes  of  such 
elements.  Item  declarations  are  usually  non-executable;  i.e.  they 
do  not  of  themselves  result  in  the  generation  of  executable  machine 
instructions.  They  provide  information  to  the  processor  useful  at 
compile  time  only,  affecting  storage  allocation  and  the  form  of 
executable  machine  instructions  that  are  generated  for  other  state¬ 
ments.  For  this  reason,  the  set  of  JCVS  tests  which  are  designed  to 
validate  the  item  definition  features,  e.g.  modules  1000-1235,  are 
inconclusive  in  themselves  as  to  what  features  are  or  are  not 
implemented  correctly.  Each  of  these  tests  upon  execution  will 
always  produce  the  "test  successful"  message,  signifying  only  that 
the  test  module  was  included  in  the  audit  program.  The  message 
has  no  significance  relative  to  the  capability  of  the  JOVIAL  pro¬ 
cessor  under  test  to  accept  and  interpret  the  feature.  Careful 
analysis  is  required  on  the  part  of  the  auditor  to  validate  the 
capability  of  the  processor  to  handle  these  features.  The  compile 
time  processor  output  might  be  helpful  to  the  auditor,  but  it,  too, 
may  be  inconclusive.  Error  flags  or  diagnostic  information,  if 
present,  indicate  things  the  processor  does  not  recognize  as  valid, 
but  the  absence  of  such  commentary  does  not  mean  that  the  features 
have  been  correctly  interpreted  and  implemented.  In  fact  a  poor 
diagnostic  or  error  detection  facility  in  the  processor  would  pro¬ 
duce  the  same  results. 

As  a  further  specific  example,  tnere  occurs  in  test  modules 
1200  and  1210  the  use  of  the  range  declaration  for  integer  and  fixed 
type  data.  According  to  the  language  specifications,  the  processor 
makes  use  of  this  information  to  provide  adequate  scaling  and  pre¬ 
cision  of  representation  of  intermediate  results.  In  order  to  test 
this  feature,  it  would  be  necessary  to  define  data  and  formulas  using 
them  which  would  demonstrate  that  scaling  of  the  intermediate  results 
was  correct.  This  has  not  been  done  in  these  or  any  other  of  the 
tests  included  in  the  Pop  File. 

In  summary,  these  tests  give  no  assurance  that  the  processor 
has  interpreted  each  feature  correctly,  since  this  can  only  be  done 
by  the  execution  of  appropriate  statements  which  reference  the  items 
and  validate  the  results. 
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PROCEDURAL  STATEMENTS 


There  are  162  test  modules  in  the  Pop  File  covering  the  features 
of  the  procedural  statements  in  the  JOVIAL  language.  These  modules 
also  reference  the  appropriate  sections  of  AFM  100-24  which  describe 
the  features  being  tested,  and  the  systematic  approach  which  has  been 
followed  seems  to  have  provided  for  the  inclusion  of  all  of  the 
essential  features  in  the  tests,  with  the  exception  that  tests  of  all 
Input/Output  features  have  been  purposely  excluded,  as  have  tests  of 
the  DIRECT: statement  (i.e.  in-line  machine  code).  These  are  con¬ 
sidered  to  be  processor  dependent  features  (cf.(2)  pp.  1,  5). 

In  general  these  tests  are  simple  and  straigh tforward  in 
design  and  adequately  validate  the  acceptance  of  the  features  being 
tested.  The  inaccessability  of  the  computed  results  and  test  values 
complicate  the  analysis  of  test  failures.  Some  of  the  test  modules 
make  use  of  features  other  than  those  under  test,  so  that  a  test 
failure  might  be  attributed  to  the  wrong  cause,  again  complicating 
the  auditor’s  analysis  of  the  failure.  While  the  JCVS  design  made 
provision  for  indicating  interdependencies  among  the  test  modules 
on  the  test  header  card,  we  found  no  instance  of  its  use  to  ensure 
that  dependent  features  would  be  tested  automatically. 

The  tests  of  arrays  and  tables  generally  use  a  snail  number  of 
entries;  most  often  2  in  the  case  of  tables,  and  sometimes  1  for  an 
array  dimension.  While  these  cases  are  consistent  with  the  standard, 
a  more  comprehensive  test  of  larger  structures  would  be  desirable  to 
demonstrate  the  capability  of  the  processor  to  handle  such  references. 
It  is  also  to  be  noted  that  many  of  the  tests  (cf  nodule  3655)  rely 
on  the  successful  referencing  of  a  single  element  of  an  array  or 
table  for  validation.  Again,  a  more  comprehensive  testing  of  many 
element  references  would  provide  better  assurance  oL  the  processor’s 
capabil i ty . 

A  detailed  analysis  of  one  of  the  more  complex  test  modules, 
module  5310,  suggests  several  ways  in  which  the  tests  night  be 
improved  from  the  standpoint  of  the  audit  function.  Module  5310 
tests  the  alternative: statement;  i.e.,  the  use  of  IFEITH. . . ORIF. . . 

The  JOVIAL  alternative: statement  provides  for  the  sequential  testing 
of  any  number  of  conditions,  each  of  which  except  the  first  occurs 
in  a  successive  or: if: clause.  Associated  with  each  condition  is  a 
single  independent  statement  which  will  be  executed  if  and  only  if 
that  condition  is  true  and  every  condition  preceding  it  is  false. 
Following  execution  of  the  associated  statement,  control  is  to  be 
returned  to  the  point  which  would  follow  the  last  or: if: clause  if 
none  of  the  set  of  alternative  conditions  were  t"uO.  Thus,  for  an 
n-condition  alternative  there  are  in  general  n+1  paths  leading  to  the 
f ol lowing  statement. 
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In  the  test  module  5310^  there  are  eight  alternative: statements, 
two  of  them  having  three  alternative  conditions,  the  others  having 
just  two  alternative  conditions  in  each.  The  expected  normal  execu¬ 
tion  of  the  program  causes  each  alternative  statement  to  be  executed 
only  once,  so  that  the  remaining  two  or  three  possible  alternative 
paths  are  not  tested  for  each  statement.  Instead,  this  test  module 
demonstrates  different  types  of  branching;  i.e.  sequences  of  alter¬ 
native  conditions  in  different  alternative: statements,  which  are 
themselves  differently  structured.  Hence,  it  does  not  conclusively 
demonstrate  that  the  processor  does  indeed  correctly  handle  the  set 
of  alternatives.  Several  of  the  conditions  in  these  alternative: 
s  ta tements  are  in  the  form  of  the  boolean  constants  0  or  1;  i.e., 
they  are  invariant.  A  sophisticated  JOVIAL  processor  with  optimiza¬ 
tion  might  recognize  these  and  generate  different  machine  code,  by¬ 
passing  the  tests  that  would  be  required  for  the  general,  variable 
alternatives.  Thus,  again,  this  method  of  testing  does  not  provide 
assurance  that  the  processor  has  correctly  implemented  or  interpreted 
the  feature.  Finally,  the  logic  of  the  test  provides  multiple  paths 
to  the  successful  conclusion  of  the  test,  so  that  a  combination  of 
errors  in  the  interpretation  of  the  alternative: statements  could 
cause  the  test  to  appear  to  be  successful.  The  points  just  enumerated 
reflect  the  primary  emphasis  of  the  JCVS  tests  on  the  demonstration 
of  the  capability  of  a  processor  to  accept  the  standard  features,  and 
the  lesser  concern  with  tne  thorough  demonstration  of  the  correctness 
of  interpretation  or  implementation  of  the  features. 

Modules  5311  and  5312  have  been  developed  to  illustrate  the  more 
thorough  testing  of  the  alternative: stacement.  In  module  5311,  a 
five  condition  alternative:  statement  is  to  be  executed  six  times, 
exercising  each  possible  branch.  Upon  completion  a  check  is  made  to 
verify  that  each  path  had  been  traversed  as  expected.  In  module  5312, 
the  same  five  conditions  are  employed,  but  internal  statement  labels 
and  branching  to  them  are  added  to  demonstrate  this  additional  capa¬ 
bility  required  within  an  alternative:  statement.  A  call  to  a  pro¬ 
cedure  is  also  introduced  to  be  executed  as  one  of  the  branches. 
Listings  of  these  two  modules  are  included  in  the  recommended  changes 
in  Appendix  I. 

PROCESSING  DECLARATIONS 

There  are  43  test  modules  devoted  to  the  JOVIAL  processing: 
declarations.  These  features  are  switches  and  user  defined  closes, 
procedures  and  functions.  Here,  too,  each  test  module  references 
the  appropriate  section  of  AFM  100-24  describing  the  feature  being 
tested.  This  set  of  tests  seems  adequate  in  terms  of  the  variety  of 
forms  and  features  in  this  category  which  are  included  in  the  tests. 
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SECTION  IV 


VALIDATION  OF  THE  JCVS  TEST  MODULES 


In  order  to  establish  validity  of  the  JCVS  test  modules  for 
performing  the  audit  functions,  they  were  reviewed  to  ensure  that: 

1.  They  were  free  of  mechanical  and  clerical  errors; 

2.  They  were  free  of  syntactic  errors,  i.e.  the  JOVIAL 
source  statements  conformed  to  the  syntactic  forms 
of  AFM  100-24; 

3.  They  were  free  of  logic  errors;  i.e.  the  results  of 
each  test  were  consistent  with  the  requirements  of 
AFM  100-24; 

4.  They  tested  a  sufficient  number  and  variety  of  the 
language  features  to  provide  an  adequate  sample  of 
the  AFM  100-24  requirements. 

The  adequacy  of  coverage  of  the  JOVIAL  language  features  is  dis¬ 
cussed  in  the  previous  section  under  the  appropriate  categories.  The 
cross  reference  list*  of  features  tested  by  the  set  of  modules  in 
the  JCVS  Pop  File  was  compared  to  the  features  defined  in  AFM  100-24 
on  a  paragraph-by- paragraph  basis.  No  glaring  omissions  were  dis¬ 
covered,  and  on  a  purely  subjective  basis  it  was  concluded  that  the 
coverage  was  adequate  for  the  intended  purpose,  with  the  reservations 
previously  cited  concerning  the  intentional  omission  of  features  from 
testing,  the  insufficiency  of  testing  of  alternatives  and  the  lack  of 
logical  rigor  of  the  tests.  The  set  of  tests  is  open-ended,  and  it 
is  presumed  that  tests  of  the  input/output  features  and  of  more  com¬ 
binations  of  features  will  be  added  in  the  future. 

A  two-phase  procedure  was  followed  in  carrying  out  the  error 
investigation.  First,  selected  test  modules  were  subjected  to  a 
desk  review  and  analysis  in  which  the  source  program  listings  were 
examined  and  checked  for  clerical  errors  and  consistency  with 
AFM  100-24.  In  some  few  instances  flow  charts  were  prepared  to 
assist  in  the  validation  of  the  logic  of  the  tests,  as  for  example 
in  the  case  of  module  5310  cited  above.  Secondly,  the  test  modules 
were  subjected  to  machine  processing  on  two  different  systems  to 
provide  an  independent  and  more  complete  verification  of  the  syntac¬ 
tic  analysis  and  correctness  of  execution  results  for  each  test. 

(4) 

The  GE-600  Line  JOVIAL  processor  was  selected  for  testing 

the  JCVS  modules  because  of  ready  access  to  the  system  on  the  GE-635 
at  the  Rome  Air  Development  Center  and  because  that  processor  was 


*See  reference  2,  p.  196-225. 
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thought  to  conform  closely  to  the  AFM  100-24  language,  the  known 
exceptions  being  the  Input/Output  features,  and  the  DUAL  and  STRING 
item:  declarations.  The  entire  JCVS  Pop  File  was  divided  into  arbit¬ 
rary  subsets  each  containing  from  50  to  70  test  modules  from  which 
were  removed  those  modules  requiring  features  known  not  to  be  implemen¬ 
ted  in  the  GE-600  Line  JOVIAL  processor.  The  subsetting  was  done  to 
avoid  exceeding  the  limitations  imposed  by  the  compiler,  to  increase 
the  probability  of  successful  compilation  and  execution  of  at  least 
some  of  the  test  modules  during  the  brief  test  period,  and  to  enable 
analyses  of  the  results  of  some  runs  to  be  made  while  others  were  in 
the  processing  queue.  A  copy  of  the  Pass/Fail  Reporter,  module  9998, 
modified  as  described  below,  was  incorporated  into  each  subset  to 
make  each  a  separate  audit  program  for  independent  compilation  and 
execution.  The  set  of  job  control  cards  shown  in  Table  I  was 
evolved  through  an  initial  series  of  eight  runs,  in  which  the  principal 
difficulties  encountered  were  in  the  setting  of  memory  limits,  the 
proper  identification  of  Input/Output  units  and  files,  and  in  the 
interpretation  of  system  error  messages.  These  control  cards  were 
attached  to  each  audit  program  deck  to  effect  the  desired  processing. 

At  least  two  runs  were  made  with  each  subset  with  the  exception  of 
the  subset  containing  modules  5190  through  6085.  That  subset  was 
run  only  once,  and  the  run  aborted  after  execution  of  all  but  five 
of  the  test  modules  in  the  set. 

After  each  run,  the  printed  output  was  examined  for  error 
diagnostic  indications  which  may  have  occurred  during  compilation. 

If  the  cause  of  the  error  could  be  determined  quickly  to  be  in  the 
test  module  and  an  obvious  correction  could  be  made,  this  was  done 
and  the  module  was  left  in  the  audit  program  for  re-processing. 
Otherwise,  the  test  module  in  question  was  removed  from  the  deck 
before  another  attempt  at  processing  was  made.  It  was  observed 
that  the  GE-600  Line  JOVIAL  processor  would  ignore  invalid  state¬ 
ments  during  the  code  generation  phase  of  compilation,  and  the  pro¬ 
gram  would  execute  with  those  parts  missing.  This  would  frequently 
result  in  the  "Test  Successful"  message  being  printed  for  that  test 
module  when  in  fact  the  test  had  not  been  performed  properly.  There¬ 
fore,  it  was  necessary  to  examine  carefully  each  run  to  verify 
successful  execution  of  the  tests. 

As  a  result  of  this  processing,  86  JCVS  test  modules  were  com¬ 
piled  and  executed  successfully;  178  modules  were  compiled  without 
generating  any  error  indications  but  were  not  executed  due  to  prior 
run  termination;  46  test  modules  were  found  to  contain  problems 
requiring  further  analysis  to  determine  the  nature  of  the  error  and 
whether  the  processor  or  the  test  module  was  at  variance  with 
AFM  100-24;  and  49  test  modules  were  not  processed.  Modifications 
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Table  I 


GE-635  Job  Control  Cards  for  JOVIAL  Compile  and  Execute  Run 


$ 

$ 

$ 

$ 

$ 

$ 


SNUMB  25460 

IDENT  EMBIL,RR0BINS0N , JCVS  ,IH519,5581 

OPTION  ERCNT/4500/, JOVIAL 

JOVIAL  NDECK 

LIMITS  08, 32000, , 10000 

INCODE  IBMEL 


**  source  deck  goes  here  ** 


$  EXECUTE 

$  ENDJOB 
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were  made  to  ten  of  the  test  modules  to  achieve  these  results. 

These  modifications  are  included  in  Appendix  I,  and  the  results  are 
detailed  in  Table  2,  Column  4. 

The  modified  Pass/Fail  Reporter  module  9998  used  the  GE-600  Line 
JOVIAL  output  statements  rather  than  requiring  a  FORTRAN  subprogram 
for  the  output  function  because  there  was  insufficient  information 
to  define  the  interface  between  the  GE-600  Line  JOVIAL  and  FORTRAN. 
While  the  GE-600  Line  JOVIAL  input/output  features  differ  from  those 
of  the  AFM  100-24  language,  they  are  used  here  only  to  perform  the 
reporting  function,  which  usage  does  not  interfere  with  the  validity 
of  the  tests  being  reported.  The  specific  modifications  consisted  of 
a  file  declaration  statement  and  a  file  opening  statement  placed  at 
the  beginning  of  the  audit  program,  and  an  output  statement  to 
replace  the  procedure  call  on  line  9998J011,  as  follows: 

FILE  PRNT  V (NORM)  R06  $  "DECLARE  PRINT  FILE" 

OUT ( I , PRNT)  $  "OPEN  PRINT  FILE" 

109998(3, PRNT, 0,AA, 7)  $  "PRINT  PASS/FAIL,  ETC." 

(3) 

The  IBM  System/360  JOVIAL  Compiler  was  selected  as  the 
second  machine  processing  test  vehicle  because  it  appeared  to  be 
the  most  complete  implementation  of  the  JOVIAL  language  and  because 
it  became  available  on  the  MITRE/360-50  system  shortly  after  the 
work  at  RADC  had  demonstrated  the  usefulness  of  machine  testing  of 
the  ~udit  programs.  The  procedure  followed  was  similar  to  that 
employed  with  the  GE-635,  except  that  only  those  test  modules  which 
had  not  executed  successfully  with  the  GE-600  Line  JOVIAL  were 
processed  on  the  system/360.  A  new  Pass/Fail  Reporter  module  0198 
previously  described  was  used  in  place  of  module  9998  in  all  /360 
JOVIAL  runs.  When,  after  each  run,  examination  of  the  printed  out¬ 
put  established  the  successful  compilation  and  execution  of  a  test 
module,  that  module  was  considered  to  be  validated  and  was  removed 
from  further  processing.  Those  test  modules  which  either  failed  to 
execute  successfully  or  had  errors  detected  in  compilation  were 
analyzed  further  in  detail.  Corrections  were  made  to  sixteen  modules 
which  were  found  at  variance  with  AFM  100-24.  The  corrected  modules 
were  processed  again  until  they,  too,  were  compiled,  executed  and 
verified  successfully.  As  a  result  of  this  processing,  178 
additional  JCVS  test  modules  were  compiled  and  executed  successfully. 
There  remain  102  test  modules  whose  status  has  not  been  resolved; 
i.e.  it  has  not  been  established  conclusively  that  these  test 
modules  are  in  conformance  to  AFM  100-24  requirements,  although  31 
of  them  did  compile  without  error  diagnostics.  A  complete  summary 
of  the  validation  results  is  presented  in  Appendix  II. 
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The  successful  compilation  and  execution  of  the  test  modules 
when  confirmed  by  examination  of  processor  generated  output,  has 
been  accepted  as  validation  of  the  modules  because  it  is  unlikely 
that  the  same  error  in  interpretation  will  be  made  in  both  the  test 
module  and  the  processor,  and  fail  to  be  noted  by  the  reviewer.  On 
the  other  hand,  the  failure  of  a  test  module  in  either  compilation 
or  execution  is  not  conclusive  evidence  that  the  test  module  is 
incorrect.  It  has  been  established  that  errors  exist  in  both  the 
GE-600  Line  JOVIAL  and  the  IBM/ 360  JOVIAL  Processors.  While  it  was 
beyond  the  scope  of  the  present  investigation  to  analyze  processor 
failures  and  make  the  necessary  corrections,  it  has  been  possible 
in  a  few  instances  to  identify  specific  processor  shortcomings,  as 
for  example,  the  IBM  processor's  failure  to  reference  correctly  a 
procedure  argument  in  making  a  boolean  comparison,  and  also  the 
failure  to  branch  correctly  on  an  index  switch.  In  these  cases  the 
generated  code  was  at  fault  although  the  interpretation  of  the  JOVIAL 
source  language  was  correct.  Aside  from  the  39  test  modules  involv¬ 
ing  dual  items,  there  are  34  test  modules  using  Table  items  which 
are  in  an  unresolved  status.  It  is  suspected,  but  not  verified, 
that  the  processor  is  responsible  for  these  failures,  and  that  the 
test  modules  are  correct. 
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SECTION  V 


SUMMARY  AND  RECOMMENDATIONS 


The  primary  objective  of  this  study  of  the  JOVIAL  Compiler 
Validation  System  was  to  establish  the  validity  of  the  set  of  tests 
with  respect  to  their  correctness  and  consistency  with  the  AFM  100-24 
Language  Specification.  Desk  analysis  and  machine  processing  with 
the  GE-600  Line  JOVIAL  and  the  IBM  System/360  J3  JOVIAL  verified 
that  260  of  the  JCVS  test  modules,  including  those  corrected,  meet 
the  criteria  for  use  in  performance  validation  by  MCS.  An  additional 
seventy- three  test  modules  are  satisfactory  on  the  basis  of  analysis 
alone,  but  could  not  be  verified  by  processing  due  to  the  absence  of 
correct  processor  implementations  of  dual  items  and  table  features. 
There  are  thirty  test  modules  which  have  not  been  validated  by  any 
means.  They,  too,  might  be  used  in  the  performance  validation,  with 
the  qualification  that  they  may  be  modified  when  and  if  discrepancies 
are  established  by  processor  rejection  and  subsequent  analysis. 

The  recommended  changes  and  corrections  to  the  test  modules  to 
achieve  these  results  are  documented  in  Appendix  I.  They  have  been 
incorporated  into  the  Population  File,  and  the  set  of  corrected 
test  modules  on  both  magnetic  tape  and  punched  cards  has  been  pro¬ 
vided  to  the  Air  Force  Directorate  of  ADPE  Selection,  together  with 
a  printed  listing  of  the  source  cards  for  the  test  modules. 

The  investigation  of  the  JCVS  test  modules  also  established  that 
there  are  areas  in  which  improvements  can  be  made  with  respect  to  the 
overall  audit  function.  It  is  recommended  that  future  work  be 
undertaken  1)  to  improve  the  rigor  and  thoroughness  of  the  tests,  as 
exemplified  by  the  MITRE  developed  modules  5311  and  5312,  and  2)  to 
provide  the  auditor  with  more  useful  information  about  the  test 
results  to  aid  in  his  analysis.  These  improvements  would  on  the 
one  hand  provide  further  assurance  of  the  correctness  of  processor 
implementation,  and  on  the  other  enhance  the  usefulness  of  the  JCVS 
in  the  MCS  performance  validation  environment. 

In  the  process  of  carrying  out  this  study  of  the  audit  programs, 
the  JCVS  auxiliary  programs  were  used  on  both  the  IBM/ 360  and 
GE-635  systems  to  prepare  the  punched  card  decks  for  processing.  The 
use  of  the  JCVS  auxiliary  programs  was  found  to  require  more  time 
and  to  be  less  convenient  than  manual  preparation  of  the  card  decks. 
It  is  recommended  that  MCS  distribute  the  JCVS  audit  programs  on 
tapes  directly  to  the  vendors  independently  of  the  JCV  System. 
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APPENDIX  I 


MODIFICATIONS  TO  THE  JCVS  TEST  MODULES 


The  corrections  to  the  JCVS  test  modules  which  were  necessary 
to  make  them  conform  to  the  AFM  100-24  requirements,  to  correct 
clerical  errors,  or  to  improve  the  performance  with  respect  to  the 
audit  function  are  cited  in  Table  II.  The  format  is  that  obtained 
by  listing  the  JOVIAL  source  language  cards  for  the  changes.  The 
four  digits  in  columns  73-76  are  the  module  numbers,  and  the  last 
three  digits  give  the  card  sequence  numbers.  These  cards  are  to 
replace  the  same  numbered  cards  in  the  Pop  File.  Also  contained  in 
this  list  are  completely  new  modules  0100,  0198,  5311  and  5312  which 
are  discussed  in  the  text. 
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TABLE  II 


JCVS  Test  Module  Corrections 


paster  mitrf  360/50  OCT 

JOVIAL  CO-pilfr  VALIDATION  systfm 
•SYSOOl*  UTILITY  'SYS002' 

•  S Y  S003 '  UTILITY  'SYS004* 

•  SY  SOO 5 •  UTILITY  #  S  Y  S006  • 

starts 

TERMS 


9  1970  JAN  ? 7  1 Q  7 1 

I B  M  160/50 

UT  ILITY 

UTILI TY 

UTILITY 


0  38  4< 


MODULE  0100  *  OUTPUT  PRINT  ROUTINE  • 

FILE  PRNT  m  1000  y  40  VIX)  6  S  *'6  IS  NA*E  defined  8  Y  •• 

* ' IMPL EMENTOR  FOP  SYSTEM  OUT- 
•'PUT  PRINTER  UNIT.  THIS 

'•STATEMENT  SHOULD  RF  REPLACED  BY  VENPO0  AS  RFQUIRFD. 


I  TFM  OUTl  H  40  S 
I T r  M  OUT  A  H  1  S 

itpm  outo  h  t  $ 

!  T  E  M  OUT  1 2  H  R  6  S 
I T  c  M  OUT  1 7  H  4  P  4  H (  )  S 

OVERLAY  OUTl  x  OUTl 2  ? OUTl i  S 
ovr  RL AY  PUT1 3  =  OUT  A  S 
I TFM  AA  H  40  $ 

TTFM  Rri  H.16  P  1 6  H  (  MODULF  TFST  )  S 
ITEM  no  H  20  P  2 OH (  )  S 

OVERLAY  AA  *  Rfi.  OUTB.DO  S 

OPEN  OUTPUT  PRNT  404UTHIS  IS  AN  AE-JCVS  TFST  *  ♦  *  )  $ 

PUTPUT  PRNT  40H(  INITIAL  TEST  OF  PRINT  PROCEDURES-  )  S 

OUTR  =  4H ( 3100)  S 

OUTERAinuTH)  S 

OUT  A  =  1 H  I N  )  S 

OUTFRBIPUTA)  S 

OUTA  =  1 H ( Y  )  S 

OU  TERBIUUTA )  S  OUTERR I  OUT  A  )  S 

OUTPUT  PRUT  4  0  H (  TpST  CnMPLF TE O-PR I  NT ED  7  LINES  *  *  *  *  I  S 

••  MOOULF  01r-«  *  three  print  routines  •• 

••OUTERR  -  FHtr  NORMAL  PRINT  ROUTINE  THAT  •' 

••  PRINTS  A  40  CHARACTER  MESSAGE  " 

•'OUTERA  -  A  ROUTINE  THAT  PUTS  A  MQOUIE  NUMBER  INTO  " 

••  A  MESSAGE  AND  PRINTS  IT  " 

''OUTERB  -  A  RO UT I NF  THAT  TESTS  A  FLAG  AND  PRINTS  TFST  PASSED  •• 

'•  OP  EAILFD  AND  THE  FLAG  •• 

PROC  OUTERR  I GG)  S 

I TFm  GG  H  40  f  I T F M  GGG  H  40  S 

BEGIN  • 'OUTERR ' ' 

GGG  =  GG  S 
OUTPUT  PRNT  GGG  S 
F  NO  '  'OUTERR '  1 
PROC  OUTfPHIbB)  $ 

I  T  EM  H  6  H  1  S 
BEGIN  •  'OUT  FR  B  '  ' 

IF  U  UT  A  FO  ] H(  Y )  S  GOTO  LM0198  S 

0UT12  *  36HI  MDOULE  TEST  FAILED  )  S 

GOTO  L NO  1 9  8  $ 

LM0198.  OUT  1 2  =  36H(  MODULE  TFST  SUCCESSFUL  )  $ 

LN019B.  OUTPUT  PRNT  OUTl  $  OUTA=lHlN)  t 


ENJ  '  ' DU  TFR  B ' ' 

PRUC  OUT  FR  A ( FE)  % 

I  TFM 

EF  H  4 

S 

IT  pm  CC  H  4  $ 

BCG I  N  "DUTERA" 

CC  = 

FF 

$ 

OUTPUT  PRNT  AA  $ 

END  '  '  OU  TF  P A  '  * 

OO'U  AOO  l 
0  001  A  no 2 
0 00 1  A  00 1 
oon ] aoo  4 

000'  A  00~> 

noon  OOn 

000] ^OO7 
01  DOC  001 
0100 JOO? 
'  'OlOOJOO  3 
'  '01o0J004 
'  '0]  00 J  006 
O' 00J007 
01  no JOO  8 
O' UOJOOO 
n l r,n JO  in 
0]  no J 01  1 
0  inn jo i 2 
0?  00J01  ■* 
or  no jo  14 
OlOOJOl* 
0100J01 6 
01  00  101  7 

0100J01 P 
oi OOJO  19 
01 00 J  02  0 
010DJ02! 
0100J027 
0100J02  3 
0 1 00  J  0?  4 
oi 00J025 
0 1 00 J02  6 
01 nRC  001 
01 QB  J002 
01 98J00? 
0198 J004 
0 1 98 J  00  5 
01 9RJ006 
01 or JOO 7 
01 9P  JOO ? 
01 98 J009 
01  QB  J  0  1  0 
01  9  8  J  01  1 
0198J012 
01 98 J  013 
01  QR  J  0 1 4 
0 1 9 B  J01  5 
0) 98 J01 6 
019R JOT  7 
0198J01  8 
0198J019 
01  9  8  J  02  0 
0198 J021 
0198J022 
0198J023 
0198 J0?4 
0 1 9  8 J  02  5 
01  98 J026 
01  98 J  02  7 
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TABLE  II  (Continued) 


•  •GOTH  ST  AT  -N  AM  F 
'  'PROCEDURE  * E RR  PRINT 


*>444 

2446 


I  snsn3 
i rnso^ 

I UQ503 
I VOS03 
I  WO  SO  3 
I XO503 
IMOS21 
IF  IA14-»o  F'j  VONI 
URI  F  I  AS  0  EO  1  4 


I  TF« 

ITcm 

ITFM 
ITEM 
I  TEM 
T  T  r  M 
I  T  Cw 


)  1 
S 
7 
IS 
1  3 
is 
1  0 


2484 
4  R 
7 
—  3 
14 
9 

1  1 


450. 
0.  , 
1  non. < 
is., 
1 400. , 
67S.  , 
..271 


.47500 

.  1 
.  1 

.99F0 
.  17SF  1 
.  8*0 

P  SO  1 


p 
p 
R 
R 
R 

R  17. 

i  GOTO  L  Y 1  4  3  0  4 
GOTO  LA521 0  $ 

IFPITh  SA5230  FO  VIA)  i  GOTO  LZS2304 
OR  1  F  S  A  5  ?  3  0  EO  VIAM  GOTO  LZ^??0  4 
LR5230.  OR  I  F  SOS  2  30  FQ  VIYESU  SC  S?  3  0=  V  I  ND  I  4 
''THIS  M00tJL  F  TFSTS  ThE  I FF  T  TH»OP I F  STATEMENTS 
*  *  M  T  X  F  O  oata  TYPES  FOR  BOOLEAN  FORMALA 

0UTS  =  4H(  1  465  I  4  TUTERAinuTRI  4 
T  TF  M  IJ^ISS  H  4  P  ^M(STFP)  4 
TARLf  TA716s  v  6  1  i 
1  6^> 

'  TL  ASSI  F  IfATT-ON  NUMBER  4.4.1.? 

I  A  7  °40 I i3 i I  -  I  4 
I  F  5  SO  2  I  4  24  I  -  3.H  4 

I  F  5  6  0  2  I  T  2  $  I -  3.14  i 

I S  5604  I 42*1  =  VIFFI  4 

I  F  '  LOCI  PA  7500.  )N0  1  0244  GO T  0  LM75004 


GOTO  LM75004 
LZ7S40.  nuTWHCn  i 

10. A6  O.01625A4 
LB1200.  OUT FPR  (otuf  )  * 


* • FRROR* • 


''CLASSIFICATION  NUMB E R  6.4.2 

''THIS  MOOULF  TESTS  T  HF  ABILITY  OF  ThP  COMpU.FR  TO 
'•USE  THE  IFFITH,  ORIF  ST  AT^mf^'TS 


• 'THI S  MODULE  WRITTEN  RY 
DUTP^4H(  5M  1  I  4 
OUTE  R A (OUT  R )  4 

ITEM  IA5M1  I 
ITEM  IP^3U  I 
I T  r m  B  AS  3 1 !  R 
LA  5  31 1  .  IFF  ITH  RA?3H  4 


ENGFL,  jp, 


71.3.15 


TF  ST  ? 


O  R  I  F 
OP  I  F 


I  A  5  31  1 
I  R  53  l  l 


FO 

GR 


16  S  P  0  4 
l  6  S  P  0  4 
P  0  4 

0  c  0  IN  I  A  5  3 l I  AS*  X 1 ♦ 16 
BA  5  31 1  =  0  4  Ff-P 
1  4  I  A  5  3  1  1  =  I  A  S  3 1  X*2  4 

4  4  BEGIN 

I  A531 1- I A531 !♦*  i 
RA  S  0 1 i * )  4 

c  NO 


OP  I  F  I  A  5  3  1  1  FQ  ?  4 


3RIF  I A  53 1  \  LS  7  4 


ns3H=  I  a  53 1  1 
I  AS  3X 1  *  I A5  3 1 1 ♦ 1 


END 

IF  TB*3 11  LS  6  4 

RFGIN  I RS31X=T  RS  3X 1 ♦!  4 
GOTO  L  A  c  3 1 1  4  CN0 

I*  IB 5  3 1 1  FQ  6  4 


051 0A001 
OM  5A  001 
1  01  OJ  02  0 
10? OJO^O 

ioi  ojom 

ioinjo3'> 
1010J033 
1  0)  0 J 03 /* 
1200 J02? 
) u 30  )00^ 
1  /*/*5  J  01  2 
1  4  5  S  J  01  1 

1  4SS.JA1  7 

I 45SJ01 8 
1465 J003 
1 465  J  004 
1  4  6  5  M  Q  0  5 
l 5^0 J01 8 
2*00J  007 
2^00J01 1 
41 22JOO? 
41  38 J028 
4 1 44 J  016 
41  56J0I  (S 
41 60 J  01 6 
4 1 75 JOO  8 
41 75 J01 0 

4  1  8  7  J  0 1  6 
4345J  014 
44SQJ016 


' 'PROCEDURES 

244 t  2404 

»  1 

4455 A001 

LPX125  . 

4  84  0  J  02  7 

L  01  200.  GOTO  l C 1 2  0  0 

4 

5 1 ^  OJ  009 

' *  I FF I TH,  no i F 

2  454 

1  1 

5311  A  00 1 

S3  1 1 J002 
5M  1  JOO* 
S3U  J  004 
5311 J005 
5311HO06 
531 1 J  00  7 
5?1 1JO08 
5  311 JOO  9 
53H  J01  0 
531 1J01 1 
s  3 1  1  J012 
5311 J013 
531  1  J01  4 
5311 JOi S 
S  3 1 1  J  01 6 
5311 JOI 7 
531  1  JOI  p 
E311J019 
5  3 1 1  J  02  0 
S311 J021 
531 1 J022 
S311J023 
5“*  1 1 J  02 4 
53UJ025 
*311 J 02  6 
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TABLE  II  (Concluded) 


BFGIN  IF  I  A  5  3 1  1  F  Q  31  S 

r  Uljn.v 

GOT P  LYS311  $  FNO 

6  ftp  JO?  P 

0UTA*1H(N1  s 

5  1 1  1  J  02  9 

GOTO  LZ5311  S 

5  3  \  1  JO  ft  0 

LY531  1  . 

OUT  A* 1H ( Y )  S 

5  31  1  JO  3  ) 

LZ531 1 . 

mJT  FR  R(  OUT  A  I  S 

5?P  J 03? 

•MFEITH,  ORIF  24  54 

,  • 

5  ft  1  2  A  00  1 

•'CLASSIFICATION  NUMBER  6.4.2 

i  • 

5  3 1 2  J  00? 

••THIS 

MDOUL  F  TFSTS  THE  ABILITY  OF  THE 

compiler  to 

•  » 

5  3 1 ?  J  00  3 

• • US  F  T  Ht  IFPITH,  ORIF  STATEMENTS 

•  • 

5ft1 2  J  004 

• 'THIS 

MODULE  WRITTEN  BY  F.  ENGEL,  JR  • 

71 .3.15 

TFST2 

*  * 

5  31  2  J  OO*' 

OU  T ft  =  4  H( 5  312  )$ 

631 2^006 

OUTFPAtOUTB)  S 

5 1 1 2  J  00  7 

ITEM  I  A  5  31  2  I  16  S  P  0  $ 

5  ftl 2 JOOP 

ITEM  IB5312  I  16  S  P  0  S 

53! ? J00c 

ITEM  ft  A  ^  ^ 1 2  ft  P  0  S 

5  31 ? J01 0 

L  A  5  31  2  • 

IFFITm  BA631 2  S  BFGIN  IA  531 2-1 A5312+16  S 

6312 J01  ) 

B A5  3 1  2sO 

s  fnd 

531 2J01 ? 

LB5312. 

OP  I F  I A  5ft  1 2  FQ  1  S  GOTO  LF5312 

i 

531  2  J01  ft 

n  p I F  I B  531 2  FQ  2  S  PA5312  S 

6  312 J01 4 

LC531  2. 

ORIF  I  A  53 1 2  FQ  3  $  BEGIN  I  A 53 1 2* I A5 31 2*4 

S 

5312  10!  c 

GOTO 

LB5312  S 

END 

631 2 J01 6 

OK  I  F  I  A  5  31 2  LS  7  S  IA6312=IA  5312  +  1  S 

5312 J01 7 

END 

5  31  2  JO  1  6 

LD531 2 . 

IF  IB5312  EQ  4  S  GOTO  LF5312  S 

53! 2 J01S 

IF  I  ft  S  3 1 2  GR  4  $  GOTO  LX5312  S 

5  31  2  J  0?  0 

I B5312-  I  B5  31 2 ♦ 1  S 

5312J02! 

GOTO  L  A  5ft 1 2  S 

5  31  2  J  02  2 

LF531 2. 

I A5  3 l 2s I A5312+2  S 

6  3 1  ft  J  02  3 

GOTO  l  C  531 2  S 

5  31  2  J  0?  4 

PROC  PA  53 1 2  % 

531  2J025 

R F G I  N  • *  PA  5312  *  • 

531? J026 

I  A5312M  A5  31  2*R  S 

6ftl?j027 

RA5ftl2=l  S 

5312  J02B 

FNO  •  •  P A  5 31  ?  #  • 

5  3 1  2  J  029 

LF531 2. 

IF  I A  5  3 1 2  FQ  31  S  GOTO  LY5312 

i 

5  31 2 JO 30 

L  X5  31 2 • 

nUT  A  *1 H  (  N  )  S 

5312J031 

GOTO  LZ 5312  S 

5312J032 

LY5312. 

OUTA=iH(Y)  s 

5312J033 

LZ631  2  . 

OUT  ER  H ( OUT  A  )  % 

631?  J034 

PR  301 2  =  15*  0A4  S 

5460 J  022 

UUTft=4H(  60*6  1  S 

6085H006 

HU  TR=4H( 6090 ) S 

6  09  OH  00  6 

OU  T  ft  =4  H( 6095  ) S 

6Q95H006 

LF5340. 

IF  NOT  (NOTtNOT  RB5340)!  S  GOTO  LY5340  S 

6126J022 

L Y  5340. 

6 1 26  J  02  f 

•  •  L  OC  A  T  I  nN 

•  • 

6735J005 

BEGIN  • • T  ft  74  2 1 • * 

6  745 J  01  7 

ITEM  IJ7421  I  12  U  S 

6745 J01 P 

L  A7421. 

IF  IK742KS1SI  EQ  12  S  GOTO  LY742!  $ 

6745J  025 

> 
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APPENDIX  II 


RESULTS  OF  THE  JCVS  TEST  MODULE  VALIDATION 


In  Table  III  the  entire  set  of  JCVS  Test  Modules  are  listed  by 
module  number,  indicating  the  results  of  machine  processing  and 
analysis  for  each  module.  The  first  column  of  this  table  indicates 
those  modules  which  were  modified  by  MITRE,  with  an  asterisk  desig¬ 
nating  those  modules  which  were  newly  created  by  MITRE.  In  the 
next  tv/o  columns  are  indicated  the  results  of  the  machine  processing 
on  the  GE-635  and  IBM/ 360,  respectively.  The  significance  of  the 
symbols  appearing  therein  is  as  follows: 

S  -  Successful  compilation  and  execution  without  error 
C  -  Compilation  without  error 

M  -  Successful  compilation  and  execution  without  error 
after  modification  by  MITRE 

F  -  Failed  in  execution  after  compilation  without  error 
N  -  Errors  detected  during  compilation 
W  -  Withheld  from  processing  for  known  deficiency  in 
processor,  e.g.  dual  item. 

A  mark  in  the  fourth  column  indicates  that  by  analysis  of  the 
source  program  and  processing  results,  if  any,  the  test  module  com¬ 
plies  with  the  requirements  of  AFM  100-24.  The  last  column  of  the 
table  indicates  those  modules  whose  status  is  unresolved;  i.e.  there 
has  been  no  confirmation  of  validity  by  successful  machine  proces¬ 
sing  of  the  module.  A  letter  D  in  this  column  identifies  test 
modules  involving  Dual  Items,  for  which  no  processor  implementation 
exists. 
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Table  III 


JCVS  Test  Module  Validation  Summary 


Module 

No.  Test  Name 


0100 

P/F  Reporter  Validat 

0198 

Pass/Fail  Reporter 

0500 

Define- 

•Preset  H  Item 

0505 

Assignment  Statement 

0510 

GOTO  Statement 

0515 

Procedure, Err  Print 

0520 

IF  Clause 

1000 

Simple 

I  terns 

1005 

Simple 

I  terns 

1010 

Simple 

I  terns 

1015 

Simple 

I  terns 

1020 

Simple 

I  terns 

1025 

Simple 

I  terns 

1030 

Simple 

I  terns 

1035 

Simple 

I  terns 

1200 

Simple 

I  terns 

1205 

Simple 

I  terns 

1210 

Simple 

I  terns 

1215 

Simple 

I  terns 

1220 

Simple 

I  terns 

1225 

Simple 

I  terns 

1235 

Simple 

I  terns 

1400 

Simple 

Items 

1405 

Simple 

I  terns 

1410 

Simple 

I  terns 

1415 

Simple 

Items 

1420 

Simple 

I  terns 

1425 

Simple 

I  terns 

1430 

Simple 

I  terns 

1435 

Simple 

I  terns 

Modified  by 

MITRE 

GE-600 

JOVIAL 

IBM/360 

JOVIAL 

Complies  by 

Analysis 

Status  not 

Resolved 

★ 

s 

★ 

s 

s 

V 

s 

V 

s 

V 

s 

V 

s 

V 

s 

V 

s 

V 

V 

M 

V 

W 

V 

D 

s 

V 

s 

V 

s 

V 

s 

V 

V 

c 

M 

V 

c 

s 

V 

c 

s 

V 

w 

V 

D 

c 

c 

V 

c 

s 

V 

c 

s 

V 

c 

s 

V 

c 

s 

V 

c 

s 

V 

w 

V 

D 

c 

s 

V 

c 

s 

V 

V 

c 

M 

V 

c 

s 

V 
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Table  III  (Continued) 

JCVS  Test  Module  Validation  Summary 


>> 

<u 

o 

CO  CO 

C  T3 

0) 

o 

0)  *H 

0) 

Module 

O  i-J 

v£)  *-3 

•H  CO 

CO  > 

M-l  W 

o  < 

ro  < 

•H  PfS 

vO  M 

\  M 

4-*  O 

No. 

Test  Name 

T3  H 

■  > 

2  £ 

6  to 

Cfl  CO 

O  W 

w  o 

PQ  O 

o  c 

4-1  0) 

X  x 

O  •“> 

M  •"> 

o  < 

CO  06 

1440 

Simple  Items 

C 

F 

V 

1445 

IFEITH,  ORIF 

v  C 

M 

v 

1450 

IFEITH,  ORIF 

C 

S 

V 

1455 

Simple  Items 

v  C 

M 

V 

1460 

IFEITH,  ORIF 

C 

F 

V 

V 

1465 

IFEITH,  ORIF 

v  N 

MF 

V 

V 

1500 

Ordinary 

Tables 

C 

S 

1505 

Ordinary 

Tables 

N 

F 

V 

V 

1510 

Ordinary 

Tables 

W 

D 

1515 

Ordinary 

Tables 

c 

S 

1520 

Ordinary 

Tables 

w 

D 

1525 

Ordinary 

Tables 

c 

F 

V 

1530 

Ordinary 

Tables 

c 

S 

1545 

Ordinary 

Tables 

c 

S 

1550 

Ordinary 

Tables 

c 

F 

V 

1555 

Ordinary 

Tables 

w 

D 

1560 

Ordinary 

Tables 

c 

S 

1565 

Ordinary 

Tables 

w 

D 

1570 

Ordinary 

Tables 

v  N 

MF 

V 

V 

1575 

Ordinary 

Tables 

c 

F 

V 

1580 

Ordinary 

Tables 

c 

S 

1585 

Ordinary 

Tables 

w 

D 

1590 

Ordinary 

Tables 

c 

S 

1595 

Ordinary 

Tables 

c 

s 

1600 

Ordinary 

Tables 

w 

D 

1605 

Ordinary 

Tables 

c 

s 

1610 

Ordinary 

Tables 

c 

s 

1615 

Ordinary 

Tables 

c 

s 

1620 

Ordinary 

Tables 

c 

s 

1625 

Ordinary 

Tables 

c 

s 
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Table  III  (Continued) 


JCVS  Test  Module  Validation  Summary 


Module 

No.  Test  Name 


>> 

X 

X 

4J 

o 

TD 

W  w 

e 

<U 

o 

<U  -r-i 

<U 

•H 

O  XI 

X  hJ 

*H  w 

w 

> 

M-i 

w 

o  < 

m  < 

*-•  ^ 

D 

•H 

X  M 

\  M 

Cu  -i 

4J 

O 

TJ 

H 

w  B 

X  > 

B  cd 

CO 

w 

O 

M 

PQ  O 

o  c 

4J 

cu 

S 

X 

O  ►-> 

M  H-) 

u  < 

C/3 

pcs 

1630 

Ordinary  Tables 

C 

S 

1635 

Ordinary  Tables 

C 

S 

1700 

Ordinary  Tables 

C 

s 

1705 

Ordinary  Tables 

C 

s 

1710 

Ordinary  Tables 

C 

s 

1715 

Ordinary  Tables 

C 

s 

1720 

Ordinary  Tables 

C 

s 

1725 

Ordinary  Tables 

C 

s 

1730 

Ordinary  Tables 

C 

F 

1735 

Ordinary  Tables 

W 

1740 

Ordinary  Tables 

C 

S 

1745 

Ordinary  Tables 

C 

S 

1750 

Ordinary  Tables 

C 

s 

1755 

Ordinary  Tables 

C 

s 

1760 

Ordinary  Tables 

C 

s 

1765 

Ordinary  Tables 

C 

s 

1770 

Ordinary  Tables 

C 

s 

1775 

Ordinary  Tables 

W 

1780 

Ordinary  Tables 

C 

s 

1785 

Ordinary  Tables 

C 

s 

1790 

Ordinary  Tables 

C 

s 

1795 

Ordinary  Tables 

C 

s 

1800 

Ordinary  Tables 

C 

s 

1805 

Ordinary  Tables 

C 

s 

1810 

Ordinary  Tables 

C 

s 
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Table  III  (Continued) 


JCVS  Test  Module  Validation  Summary 


Module 

No.  Test  Name 


>> 

4J 

O 

to  to 

C  T> 

0) 

o 

<D-H 

0) 

♦H 

o 

\D 

*H  CO 

to  > 

VM  W 

o  < 

m  < 

>s 

0  ^ 

*h  a; 

vO  M 

\  M 

di— i 

4J  O 

T3  H 

'  > 

S  > 

6  td 

td  to 

O  M 

w  o 

«  o 

o  c 

4-J  0) 

M  ►“> 

CO  aS 

1815 

Ordinary  Tables 

N 

w 

D 

1820 

Ordinary  Tables 

C 

s 

1825 

Ordinary  Tables 

C 

s 

1830 

Ordinary  Tables 

C 

s 

1835 

Ordinary  Tables 

C 

s 

1840 

Ordinary  Tables 

c 

s 

1845 

Ordinary  Tables 

c 

s 

1850 

Ordinary  Tables 

c 

s 

1855 

Ordinary  Tables 

c 

F 

V 

1860 

Ordinary  Tables 

w 

D 

1900 

Ordinary  Tables 

c 

S 

1905 

Ordinary  Tables 

c 

s 

1910 

Ordinary  Tables 

c 

s 

1915 

Ordinary  Tables 

c 

s 

1920 

Ordinary  Tables 

c 

s 

1925 

Ordinary  Tables 

c 

s 

1930 

Ordinary  Tables 

c 

s 

1935 

Ordinary  Tables 

c 

s 

1940 

Ordinary  Tables 

c 

s 

1945 

Ordinary  Tables 

w 

D 

2500 

Defined  Tables 

v  N 

M 

V 

2505 

Defined  Tables 

N 

F 

V 

2510 

Defined  Tables 

N 

N 

V 

2515 

Defined  Tables 

N 

N 

V 

2520 

Defined  Tables 

N 

F 

V 
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Table  III  (Continued) 

JCVS  Test  Module  Validation  Summary 


Module 


xj 

<D 


W 

& 

H 


O  J 
o  < 

vO  M 

1  > 


o 

ro  < 
£  > 


w 

<D 


w 


X! 
OJ 
> 
*— i 

o 

w 


No. 

Test  Name 

OH  WO 

S  S  OH 

«  O  O  c 

MX>  a  < 

4-1  0) 

CO  pcs 

2525 

Defined  Tables 

c 

s 

2530 

Defined  Tables 

w 

D 

2535 

Defined  Tables 

N 

F 

V 

2540 

Defined  Tables 

N 

N 

V 

3500 

Arrays 

C 

S 

3505 

Arrays 

c 

S 

3510 

Arrays 

c 

S 

3515 

Arrays 

c 

S 

3520 

Arrays 

c 

S 

3525 

Arrays 

c 

N 

V 

3530 

Arrays 

w 

D 

3535 

Arrays 

c 

S 

3540 

Arrays 

c 

S 

3545 

Arrays 

F 

F 

V 

3600 

Defined  Tables 

w 

N 

V 

3605 

Defined  Tables 

w 

N 

V 

3610 

Defined  Tables 

w 

N 

V 

3615 

Defined  Tables 

w 

N 

V 

3620 

Defined  Tables 

w 

N 

V 

3625 

Defined  Tables 

w 

N 

V 

3630 

Defined  Tables 

w 

S 

3635 

Defined  Tables 

w 

D 

3640 

Defined  Tables 

w 

N 

V 

3645 

Defined  Tables 

w 

N 

V 

3650 

Defined  Tables 

w 

N 

V 
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Table  III  (Continued) 

JCVS  Test  Module  Validation  Summary 


>> 

Si 

03 

Module  w 

No.  Test  Name  'g  H 

5C  S 


3655 

Defined  Tables 

W 

N 

V 

3660 

Defined  Tables 

W 

N 

V 

3990 

Switches 

s 

3905 

Switches 

s 

3910 

Swi  tches 

s 

3915 

Switches 

F 

F 

V  V 

3920 

Switches 

F 

S 

3925 

Swi tches 

W 

D 

3930 

Switches 

s 

3935 

Switches 

s 

3940 

Switches 

s 

3945 

Switches 

s 

3950 

Switches 

s 

3955 

Switches 

s 

3960 

Swi  tches 

s 

3965 

Switches 

N 

F 

V  V 

4000 

BIT 

c 

S 

4005 

BIT 

c 

S 

4010 

BIT 

c 

S 

4015 

BIT 

c 

S 

4080 

BYTE 

c 

S 

4085 

BYTE 

c 

S 

4090 

BYTE 

c 

S 

4095 

BYTE 

c 

S 

4120 

ENTRY  and  ENT 

c 

F 

V 

>> 

4-1 

o 

m  co 

c 

T3 

o 

0)  ^4 

03 

o 

vO 

•H  co 

CO 

> 

o 

< 

CO 

< 

r* 

3 

r“4 

vO 

M 

M 

g-  «-* 

4J 

o 

1 

> 

g 

> 

g  co 

co 

CO 

w 

o 

PQ 

O 

O  c 

4-1 

03 

o 

•“> 

M 

u  < 

CO 

P£ 
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Table  III  (Continued) 


JCVS  Test  Module  Validation  Summary 


Module 

No.  Test 

Name 

Modified  by 

MITRE 

GE-600 

JOVIAL 

IBM/ 360 

JOVIAL 

Complies  by 

Analysis 

Status  not 

Resolved 

4122 

ENTRY 

and 

ENT 

V 

N 

MF 

V 

4124 

ENTRY 

and 

ENT 

C 

N 

V 

4126 

Entry 

and 

Ent 

c 

F 

V 

4128 

Entry 

and 

Ent 

c 

F 

v 

4130 

Entry 

and 

Ent 

c 

F 

V 

4132 

Entry 

and 

Ent 

c 

F 

V 

4134 

Entry 

and 

Ent 

c 

F 

V 

4136 

Entry 

and 

Ent 

F 

V 

4138 

Entry 

and 

Ent 

V 

F 

MF 

V 

V 

4140 

Entry 

and 

Ent 

C 

S 

4142 

Entry 

and 

Ent 

C 

S 

4144 

Entry 

and 

Ent 

V 

N 

M 

V 

4146 

Entry 

and 

Ent 

C 

S 

4148 

Entry 

and 

Ent 

C 

S 

4150 

Entry 

and 

Ent 

c 

s 

4152 

Entry 

and 

Ent 

c 

s 

4154 

Entry 

and 

Ent 

c 

s 

4156 

Entry 

and 

Ent 

V 

N 

M 

V 

4158 

Entry 

and 

Ent 

c 

S 

4160 

Entry 

and 

Ent 

V 

N 

M 

V 

4162 

ENTRY 

and 

ENT 

C 

S 

4175 

PRIME 

LOC 

N 

N 

V 

4178 

PRIME 

LOC 

V 

C 

NM 

V 

V 

4181 

PRIME 

LOC 

N 

N 

V 

4184 

PRIME 

LOC 

C 

N 

V 

4187 

PRIME 

LOC 

V 

N 

M 

V 

4190 

PRIME 

LOC 

S 

4193 

PRIME 

LOC 

F 

N 

V 

4196 

PRIME 

LOC 

N 

N 

V 

4199 

PRIME 

LOC 

N 

N 

V 
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Table  III  (Continued) 


JCVS  Test  Module  Validation  Summary 


>> 


Module 

No.  Test  Name 


’O 

0) 

•H 

O 

hJ 

o 

VO  hJ 

3.2 

*  cn 

s  no 

'O 

0) 

> 

4-1 

w 

o 

< 

ro  < 

rv 

D 

H 

Pd 

vO 

^  M 

§« 

•U 

o 

T3 

H 

l 

> 

2  > 

CC 

cn 

O 

M 

w 

o 

PP  O 

■u 

0) 

2 

2 

o 

M  *") 

CO 

Pd 

4200 

NENT 

S 

4205 

NENT 

S 

4210 

NENT 

S 

4215 

NENT 

S 

4220 

NENT 

S 

4225 

NENT 

S 

4230 

NENT 

S 

4235 

NENT 

N 

w 

4300 

NWDSEN 

S 

4305 

NWDSEN 

S 

4310 

NWDSEN 

s 

4315 

NWDSEN 

s 

4340 

ODD 

s 

4345 

ODD 

v  F 

M 

V 

4350 

ODD 

S 

4390 

ALL 

S 

4450 

Procedures 

v  N 

M 

V 

4455 

Procedures 

C 

s 

4460 

Procedures 

C 

s 

4465 

Procedures 

c 

s 

4470 

Procedures 

c 

s 

4475 

Procedures 

c 

s 

4480 

Procedures 

N 

s 

4485 

Procedures 

N 

s 

4490 

Procedures 

c 

s 

4495 

Procedures 

c 

s 

4470 

Procedures 

c 

s 

V 

4475 

Procedures 

c 

s 

V 

4780 

Procedures 

c 

s 

V 

4785 

Procedures 

c 

s 

V 
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Table  III  (Continued) 


JCVS  Test  Modules  Validation  Summary 


Module 


No. 

Test  Name 

4800 

Functions 

4805 

Functions 

4810 

Functions 

4815 

Func  tions 

4820 

Functions 

4825 

Functions 

4830 

Functions 

4835 

Functions 

4840 

Functions 

4845 

Functions 

5100 

Functions 

5105 

Functions 

5110 

Functions 

5115 

Func  tions 

5120 

Functions 

5160 

Functions 

5180 

RETURN 

5185 

RETURN 

5190 

RETURN 

5280 

IF  Clause 

5310 

IFEITH,  ORIF 

5311 

IFEITH,  ORIF 

5312 

IFEITH,  ORIF 

5340 

FOR  Loops 

5342 

FOR  Loops 

5344 

FOR  Loops 

5346 

FOR  Loops 

5348 

FOR  Loops 

5350 

FOR  Loops 

5352 

FOR  Loops 

Modified  by 

MITRE 

GE-600 

JOVIAL 

IBM/ 360 

JOVIAL 

Complies  by 

Analysis 

c 

s 

c 

s 

c 

s 

c 

s 

c 

s 

N 

N 

N 

s 

c 

s 

V 

c 

M 

V 

w 

c 

s 

V 

c 

s 

V 

c 

s 

V 

c 

s 

V 

c 

s 

V 

V 

N 

M 

V 

s 

s 

s 

s 

s 

V 

* 

s 

V 

* 

s 

V 

S 

S 


s 

s 

s 

s 

s 


o 

e 

0) 

w  > 

2  r-i 

4J  O 
co  w 
4-J  d) 

m  d£ 


D 
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Table  III  (Continued) 


JCVS  Test  Module  Valiation  Summary 


4 


Module 

No.  Test  Name 


>> 

X 

X 

XJ 

0 

TJ 

CO  CO 

c 

XJ 

0) 

o 

0 i 

0) 

o  iJ 

x 

x 

'r4  CO 

CO 

> 

4-f  W 

o  < 

fO 

< 

— <  >> 

3 

•H  p5 

X  M 

M 

O-H— < 

xj 

O 

T3  H 

'  > 

£ 

> 

E  03 

03 

CO 

O  H 

w  o 

O 

o  c 

XJ 

0) 

x  2 

o  •-> 

M 

•“> 

o  < 

CO 

cc 

5354  FOR  Loops  S 
5356  FOR  Loops  S 
5358  FOR  Loops  S 
5360  FOR  Loops  S 
5362  FOR  Loops  S 

5364  FOR  Loops  S 
5366  FOR  Loops  S 
5368  FOR  Loops  S 
5370  FOR  Loops  S 
5372  FOR  Loops  S 


5390  Loop  Control 
5392  Loop  Control 
5394  Loop  Control 
5396  Loop  Control 
5400  Numeric  Expression 


S 

S 

S 

S 

S 


5405 

5410 

5415 

5420 

5425 


Numeric  Expression 
Numeric  Expression 
Numeric  Expression 
Numer ic  Express ion 
Numeric  Expression 


F 

S 

S 

s 

s 


F 


v 


5430 

5435 

5440 

5445 

5450 


Numeric  Expression 
Numer ic  Express  ion 
Numeric  Expression 
Numeric  Expression 
Numeric  Expression 


S 

S 

S 

S 

S 


5455 

5460 

5465 

5470 

5475 


Numeric  Expression 
Numeric  Expression 
Numeric  Expression 
Numeric  Expression 
Numeric  Expression 


S 

F 

S 

F 

S 


N 

S 


v 
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Table  III  (Continued) 


JCVS  Test  Module  Validation  Summary 


Module 

No.  Test  Name 


5480 

Numeric 

Expressions 

5485 

Numeric 

Expressions 

5490 

Numeric 

Expressions 

5495 

Numeric 

Expressions 

5500 

Numeric 

Expressions 

5505 

Numeric 

Expressions 

5510 

Numeric 

Expressions 

5515 

Numeric 

Expressions 

5520 

Numeric 

Expressions 

5525 

Numeric 

Expressions 

5530 

Numeric 

Expressions 

5535 

Numeric 

Expressions 

5540 

Numeric 

Expressions 

5545 

Numeric 

Expressions 

5800 

Simple 

Comparisons 

5805 

Simple  1 

Comparisons 

5810 

Simple  1 

Comparisons 

5815 

Simple 

Comparisons 

5820 

Simple 

Comparisons 

5825 

Simple 

Comparisons 

5830 

Simple 

Comparisons 

5835 

Simple 

Comparisons 

5840 

Simple 

Comparisons 

6000 

Chained 

Comparisons 

6005 

Chained 

Comparisons 

6010 

Chained 

Comparisons 

6015 

Chained 

Comparisons 

6020 

Chained 

Comparisons 

6025 

Chained 

Comparisons 

6030 

Chained 

Comparisons 

Modified  by 

MITRE 

GE-600 

JOVIAL 

IBM/ 360 

JOVIAL 

Complies  by 

Analysis 

Status  not 

Resolved 

s 

w 

D 

w 

D 

w 

D 

w 

D 

w 

D 

w 

D 

w 

D 

w 

D 

w 

D 

w 

D 

w 

D 

w 

D 

w 

D 

S 

S 

s 

W  D 

s 

s 

s 

s 

s 

N  S 

N  S 

c  s 

W  D 

C  S 

C  S 

N  S 
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Table  III  (Continued) 

JCVS  Test  Module  Validation  Summary 


Module 


No. 

Test  Name 

6085 

Boolean  Expression 

6090 

Boolean  Expression 

6095 

Boolean  Expression 

6100 

Boolean  Expression 

6105 

Boolean  Expression 

6100 

Boolean  Expression 

6115 

Boolean  Expression 

6120 

Boolean  Expression 

6125 

Boolean  Expression 

6130 

Boolean  Expression 

6135 

Boolean  Expression 

6200 

Assignment 

6400 

Exchange 

6405 

Exchange 

6410 

Exchange 

6415 

Exchange 

6420 

Exchange 

6425 

Exchange 

6430 

Exchange 

6435 

Exchange 

6440 

Exchange 

6445 

Exchange 

6450 

Exchange 

6455 

Exchange 

6460 

Exchange 

6465 

Exchange 

6470 

Exchange 

6475 

Exchange 

6500 

DEFINE 

6600 

LIKE 

Modified  by 

MITRE 

GE-600 

JOVIAL 

IBM/ 360 

JOVIAL 

Complies  by 

Analysis 

V 

N 

N 

V 

C 

M 

V 

V 

C 

M 

V 

c 

S 

c 

N 

c 

S 

N 

S 

N 

s 

V 

N 

M 

V 

N 

s 

C 

F 

w 

V 

c 

S 

c 

S 

c 

S 

w 

c 

S 
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Table  III  (Concluded) 


JCVS  Test  Module  Validation  Summary 
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